Pseudo-Marginal MCMC for Parameter Estimation in α-Stable Distributions

The α-stable distribution is very useful for modelling data with extreme values and skewed behaviour. The distribution is governed by two key parameters, tail thickness and skewness, in addition to scale and location. Inferring these parameters is difficult due to the lack of a closed form expression of the probability density. We develop a Bayesian method, based on the pseudo-marginal MCMC approach, that requires only unbiased estimates of the intractable likelihood. To compute these estimates we build an adaptive importance sampler for a latentvariable-representation of the α-stable density. This representation has previously been used in the literature for conditional MCMC sampling of the parameters, and we compare our method with this approach.This is the author accepted manuscript. The final version is available from Elsevier via http://dx.doi.org/10.1016/j.ifacol.2015.12.17

Fully Bayesian inference for α-stable distributions using a Poisson series representation

In this paper we develop an approach to Bayesian Monte Carlo inference for skewed α-stable distributions. Based on a series representation of the stable law in terms of infinite summations of random Poisson process arrival times, our framework leads to a simple representation in terms of conditionally Gaussian distributions for certain latent variables. Inference can therefore be carried out straightforwardly using techniques such as auxiliary variables versions of Markov chain Monte Carlo (MCMC) methods. The Poisson series representation (PSR) is further extended to practical application by introducing an approximation of the series residual terms based on exact moment calculations. Simulations illustrate the proposed framework applied to skewed α-stable simulated and real-world data, successfully estimating the distribution parameter values and being consistent with other (non-Bayesian) approaches. The methods are highly suitable for incorporation into hierarchical Bayesian models, and in this case the conditionally Gaussian structure of our model will lead to very efficient computations compared to other approaches.Godsill acknowledges partial funding for the work from the EPSRC BTaRoT project EP/K020153/1, and Tatjana Lemke acknowledges PhD funding from Fraunhofer ITWM, Kaiserslautern.This is the author accepted manuscript. The final version is available from Elsevier via http://dx.doi.org/10.1016/j.dsp.2015.08.01

Simulated convergence rates with application to an intractable α-stable inference problem

© 2017 IEEE. We report the results of a series of numerical studies examining the convergence rate for some approximate representations of α-stable distributions, which are a highly intractable class of distributions for inference purposes. Our proposed representation turns the intractable inference for an infinite-dimensional series of parameters into an (approximately) conditionally Gaussian representation, to which standard inference procedures such as Expectation-Maximization (EM), Markov chain Monte Carlo (MCMC) and Particle Filtering can be readily applied. While we have previously proved the asymptotic convergence of this representation, here we study the rate of this convergence for finite values of a truncation parameter, c. This allows the selection of appropriate truncations for different parameter configurations and for the accuracy required for the model. The convergence is examined directly in terms of cumulative distribution functions and densities, through the application of the Berry theorems and Parseval theorems. Our results indicate that the behaviour of our representations is significantly superior to that of representations that simply truncate the series with no Gaussian residual term

Considering discrepancy when calibrating a mechanistic electrophysiology model

Uncertainty quantification (UQ) is a vital step in using mathematical models and simulations to take decisions. The field of cardiac simulation has begun to explore and adopt UQ methods to characterize uncertainty in model inputs and how that propagates through to outputs or predictions; examples of this can be seen in the papers of this issue. In this review and perspective piece, we draw attention to an important and under-addressed source of uncertainty in our predictions—that of uncertainty in the model structure or the equations themselves. The difference between imperfect models and reality is termed model discrepancy, and we are often uncertain as to the size and consequences of this discrepancy. Here, we provide two examples of the consequences of discrepancy when calibrating models at the ion channel and action potential scales. Furthermore, we attempt to account for this discrepancy when calibrating and validating an ion channel model using different methods, based on modelling the discrepancy using Gaussian processes and autoregressive-moving-average models, then highlight the advantages and shortcomings of each approach. Finally, suggestions and lines of enquiry for future work are provided. This article is part of the theme issue ‘Uncertainty quantification in cardiac and cardiovascular modelling and simulation’

Approximate simulation of linear continuous time models driven by asymmetric stable Lévy processes

In this paper we extend to the multidimensional case the modified Poisson series representation of linear stochastic processes driven by α-stable innovations. The latter has been recently introduced in the literature and it involves a Gaussian approximation of the residuals of the series, via the exact characterization of their moments. This allows for Bayesian techniques for parameter or state inference that would not be available otherwise, due to the lack of a closed-form likelihood function for the α-stable distribution. Simulation results are presented to validate the introduced extension and the quality of the approximation of the distribution. Finally, we show an example of generation from the process

Pseudo-Marginal MCMC for Parameter Estimation in α-Stable Distributions

The α-stable distribution is very useful for modelling data with extreme values and skewed behaviour. The distribution is governed by two key parameters, tail thickness and skewness, in addition to scale and location. Inferring these parameters is difficult due to the lack of a closed form expression of the probability density. We develop a Bayesian method, based on the pseudo-marginal MCMC approach, that requires only unbiased estimates of the intractable likelihood. To compute these estimates we build an adaptive importance sampler for a latentvariable- representation of the α-stable density. This representation has previously been used in the literature for conditional MCMC sampling of the parameters, and we compare our method with this approach

Convergence results for tractable inference in α-stable stochastic processes

The α-stable distribution is highly intractable for inference because of the lack of a closed form density function in the general case. However, it is well-established that the α-stable distribution admits a Poisson series representation (PSR) in which the terms of the series are a function of the arrival times of a unit rate Poisson process. In our previous work, we have shown how to carry out inference for regression models using this series representation, which leads to a very convenient conditionally Gaussian framework, amenable to tractable Gaussian inference procedures. The PSR has to be truncated to a finite number of terms for practical purposes. The residual error terms have been approximated in our previous work by a Gaussian distribution, and we have recently shown that this approximation can be justified through a Central Limit Theorem (CLT). In this paper we present a new and exact characterisation of the first and second moments of the residual series over finite time intervals for the unit rate Poisson process, correcting a previous version that was only true in the infinite time limit. This enables us to test through simulation the rapid convergence of the residual terms to a Gaussian distribution of the Poisson series residual. We test this convergence using both Q-Q plots and the classical Kolmogorov-Smirnov test of Gaussianity

The Lévy State Space Model

In this paper we introduce a new class of state space models based on shot-noise simulation representations of nonGaussian Lévy-driven linear systems, represented as stochastic differential equations. In particular a conditionally Gaussian version of the models is proposed that is able to capture heavy-tailed non-Gaussianity while retaining tractability for inference procedures. We focus on a canonical class of such processes, the α-stable Lévy processes, which retain important properties such as self-similarity and heavy-tails, while emphasizing that broader classes of non-Gaussian Lévy processes may be handled by similar methodology. An important feature is that we are able to marginalise both the skewness and the scale parameters of these challenging models from posterior probability distributions. The models are posed in continuous time and so are able to deal with irregular data arrival times. Example modelling and inference procedures are provided using Rao-Blackwellised sequential Monte Carlo applied to a two-dimensional Langevin model, and this is tested on real exchange rate data

Fully Bayesian inference for α-stable distributions using a Poisson series representation

In this paper we develop an approach to Bayesian Monte Carlo inference for skewed α-stable distributions. Based on a series representation of the stable law in terms of infinite summations of random Poisson process arrival times, our framework leads to a simple representation in terms of conditionally Gaussian distributions for certain latent variables. Inference can therefore be carried out straightforwardly using techniques such as auxiliary variables versions of Markov chain Monte Carlo (MCMC) methods. The Poisson series representation (PSR) is further extended to practical application by introducing an approximation of the series residual terms based on exact moment calculations. Simulations illustrate the proposed framework applied to skewed α-stable simulated and real-world data, successfully estimating the distribution parameter values and being consistent with other (non-Bayesian) approaches. The methods are highly suitable for incorporation into hierarchical Bayesian models, and in this case the conditionally Gaussian structure of our model will lead to very efficient computations compared to other approaches

Nonasymptotic Gaussian Approximation for Inference with Stable Noise

The results of a series of theoretical studies are reported, examining the convergence rate for different approximate representations of alpha -stable distributions. Although they play a key role in modelling random processes with jumps and discontinuities, the use of alpha -stable distributions in inference often leads to analytically intractable problems. The LePage series, which is a probabilistic representation employed in this work, is used to transform an intractable, infinite-dimensional inference problem into a finite-dimensional (conditionally Gaussian) parametric problem. A major component of our approach is the approximation of the tail of this series by a Gaussian random variable. Standard statistical techniques, such as Expectation-Maximization (EM), Markov chain Monte Carlo, and Particle Filtering, can then be readily applied. In addition to the asymptotic normality of the tail of this series, we establish explicit, nonasymptotic bounds on the approximation error. Their proofs follow classical Fourier-analytic arguments, using Esséen's smoothing lemma. Specifically, we consider the distance between the distributions of: (i) the tail of the series and an appropriate Gaussian; (ii) the full series and the truncated series; and (iii) the full series and the truncated series with an added Gaussian term. In all three cases, sharp bounds are established, and the theoretical results are compared with the actual distances (computed numerically) in specific examples of symmetric alpha -stable distributions. This analysis facilitates the selection of appropriate truncations in practice and offers theoretical guarantees for the accuracy of resulting estimates. One of the main conclusions obtained is that, for the purposes of inference, the use of a truncated series together with an approximately Gaussian error term has superior statistical properties and is likely a preferable choice in practice